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A method for indicating the location of time dependent video hot links (1080, 
2080) to a user, comprising the steps of: displaying a motion video presentation 
on a first portion (1010, 2010) of a display device (1000, 2000), the video 
presentation including a hot link region (1080, 2080) which can be selected by a 
user to link to different content; displaying an indication (1060, 2009, 2100) of the 
presence of the hot link region (1080, 2080) in the video presentation. The 
method may comprise the further step of altering a visual attribute of at least a 
portion of the hot link region (1080, 2080) when a user manipulate cursor is 
within the first portion. The method may comprise the further step of displaying 
an indication (2009, 2100) of the presence of the hot link region (1080, 2080) in 
the video presentation on a second portion (1030, 2030) of the display screen 
(1000, 2000). 



Data supplied from the esp@cenet database - http://ep.espacenet.com 



This Patent PDF Generated by Patent Fetcher(TM), a service of Patent Logistics, LLC 



(19) 



Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 



(12) 



(n) EP 0 840 241 A1 

EUROPEAN PATENT APPLICATION 



(43) Date of publication: 

06.05.1998 Bulletin 1998/19 

(21) Application number: 97308465.0 

(22) Date of filing: 23.10.1997 



(51) Intel * G06F 17/30 



(84) Designated Contracting States: 


(72) Inventors: 


AT BE CH DE DK ES Fl FR GB GR IE IT LI LU MC 


• Chen, Jeane 


NL PT SE 


, Chappaqua, New York 1 051 4 (US) 


Designated Extension States: 


Feig, Ephraim 


AL LT LV RO SI 


Chappaqua, New York 10514 (US) 




• Zhang, Liang Jie 


(30) Priority: 01.11.1996 CN 96122685 


White Plains, New York 10603 (US) 


(71) Applicant: INTERNATIONAL BUSINESS 


(74) Representative: Ling, Christopher John 


MACHINES CORPORATION 


IBM United Kingdom Limited, 


Armonk, NY 10504 (US) 


intellectual Property Department, 




Hursley Park 




Winchester, Hampshire S021 2JN (GB) 



(54) A method for indicating the location of video hot links 



(57) A method for indicating the location of time de- 
pendent video hot links (1 080, 2080) to a user, compris- 
ing the steps of: displaying a motion video presentation 
on a first portion (1010, 2010) of a display device (1000, 
2000), the video presentation including a hot link region 
(1080, 2080) which can be selected by a user to link to 
different content; displaying an indication (1060, 2009, 
2100) of the presence of the hot link region (1 080, 2080) 



in the video presentation. The method may comprise the 
further step of altering a visual attribute of at least a por- 
tion of the hot link region (1080, 2080) when a user ma- 
nipulate cursor is within the first portion. The method 
may comprise the further step of displaying an indication 
(2009, 21 00) of the presence of the hot link region (1 080, 
2080) in the video presentation on a second portion 
(1030, 2030) of the display screen (1000, 2000). 
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Description 

Field of the Invention 

The present invention relates to information 
processing of computer networks, more specifically, to 
a method for indicating the location of video hot links. 

Background of the Invention 

Hypermedia is a term used to describe the fusion 
of two other new technologies: multimedia and hyper- 
text. Multimedia refers to forms of information containing 
text, images, graphics, audio and video. A hypertext 
document is one which is linked to other documents via 
hyperlinks. A hyperlink often appears in a hypertext doc- 
ument as a piece of highlighted text. The text is usually 
a word or phrase describing something of which a user 
might want further information. When the user activates 
the hyperlink, typically by clicking on it using a mouse, 
the user view is changed so as to show the linked doc- 
ument, which typically contains more information on the 
highlighted word or phrase concerned. Hyperlinks make 
it easy to follow cross-references between documents. 
Hypermedia documents are hypertext documents with 
multimedia capabilities. The regions on the screen 
which are active hyperlinks are called hot-links. 

Nowadays, most people are familiar with the appli- 
cation of hypertext by using a mouse to click on hot-links 
on computer displays of homepages from the World 
Wide Web (the Web) on the Internet. Data on the Web 
is located via URLs. URL stands for Uniform Resource 
Locator. It is a draft standard for specifying as object on 
the Internet. It specifies access methods and the loca- 
tion of the files. Documents on the Web are written in a 
simple markup language called HTML, which stands for 
Hypertext Markup Language. File formats of data on the 
Web are specified as MIME formats; MIME stands for 
"Multipurpose Internet Mail Extensions". (Reference: ht- 
tp://www. oac.uci.edu/indiv/ehood/MIME/ MIME.html). 
Examples of file formats on the Web are .au (probably 
the most common audio format), .html (HTML files), .jpg 
(JPEG encoded images), .mid (Midi music format), . 
mpg (MPEG encoded video), and .ps (postcript files) . 

While presently hypertext technology is most com- 
mon in text and image media, it is beginning to also ap- 
pear in animation and video. HyperVideo is the name 
for video augmented with hyperlinks. NEC corporation 
has demonstrated to Newsbytes such a system, named 
video hypermedia system, that will bring the point and 
click capabilities of hypertext to full motion video (NEC's 
Video Hypertext System, Newsbytes News Network, Ju- 
ly 31, 1995.). 

HyperCafe is an experimental hypermedia proto- 
type, developed as an illustration of a general hyper- 
video system (Nitin "Nick" Sawhney, David Balcom and 
Ian Smith, HyperCafe: Narrative and Aesthetic Proper- 
ties of Hypervideo, Hypertext 96: Seventh ACM Confer- 



ence on Hypertext (Recipient of the first Engelbart Best 
Paper Award at Hypertext '96, March 20, 1 996), http:// 
silver.skiles.gatech.edu/gallery/hypercafe/ 
HT96_Talk/). This program places the user in a virtual 
5 cafe, composed primarily of digital video clips of actors 
involved in fictional conversations in the cafe. HyperCa- 
fe allows the user to follow different conversations, and 
offers dynamic opportunities of interaction via temporal, 
spatio-temporal and textual links to present alternative 

10 narratives. 

VldeoActive is an authoring tool for the creation of 
interactive movies (HyperVideo Authoring Tool (User 
Notes), http://ephyx.com/, Pre-Release version, Feb. 
1 996). It uses the HyperVideo technology to include hot- 

is links in digital video files. The tool allows one to prepare 
video clips with the hot-link information and then to link 
them with other types of media. 

Hot links in hypertext files are highlighted presently 
by way of color variation. When normal hypertexts are 

20 black, for instance, the hypertexts containing hot links 
are of another color (e.g. blue). In this way, the users 
can learn whether there exists a hot link in the hypertexts 
through changes of the colors. However, color is impor- 
tant information in other hypermedia (image and video) 

25 fjies, hence the user might get a distorted image and 
video if the same method as mentioned above were still 
used to indicate whether there exist hot links in the im- 
age and video files. In the case where there is. a red 
flower in a certain frame of a video, for instance, the in- 

30 formation contained in the video itself will evidently dis- 
torted if the hot link on the red flower is indicated directly 
by way of color variation. Therefore, it is undesirable to 
directly adopt the method of color variation in hyperme- 
dia (such as video) files. In the current hypervideo demo 

35 systems, the mouse cursor is often moved into a video 
display window and is moved around continuously with- 
in the window to find out whether there exist hot links in 
the video. For instance, when the cursor shape chang- 
es, it shows that there is a hot link. The method for seek- 

40 ing hot links contained in the video by way of a mouse 
cursor's roaming within the video window is very incon- 
venient. 

The objective of the invention is to provide a method 
for indicating the location of time dependent video hot 
45 links to a user, including a method for indicating the pres- 
ence and the location of a hot link. 

Disclosure of the Invention 

so Accordingly, the present invention provides a meth- 
od for indicating the location of time dependent video 
hot links to a user, comprising the steps of: displaying a 
motion video presentation on a first portion of a display 
device, the video presentation including a hot link region 

ss which can be selected by a user to link to different con- 
tent; displaying an indication of the presence of the hot 
link region in the video presentation. 

The method of the invention may comprise the fur- 
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ther step of altering a visual attribute of at least a portion 
of the hot link region when a user manipulable cursor is 
within the first portion. 

The method of the invention may comprise the fur- 
ther step of displaying an indication of the presence of 
the hot link region in the video presentation on a second 
portion of the display screen. 

With the method of the present invention, it is pos- 
sible to indicate the presence and location of a hot link 
without the mouse cursor roaming or intrusion into the 
active video window. That is to say, a user can view in 
a non -intrusive mode, while he obtains the information 
about the presence and the corresponding location (if 
present) of a hot link. 

Brief Description of the Drawings 

The invention will now be described, by way of ex- 
ample only, with reference to the accompanying draw- 
ings, in which: 

Fig. 1 shows schematically a first embodiment of the 
present invention; and 

Fig. 2 shows schematically a second embodiment 
of the present invention. 

Detailed Description of the Invention 

Fig. 1 shows schematically a first embodiment of 
the invention. In this embodiment, video is being dis- 
played on a computer monitor 1000. This monitor dis- 
plays data (text, video) which is generated by a compu- 
ter 1001, to which are also attached a keyboard 1002 
and a mouse 1 003 in standard configuration. Speakers 
are attached to the computer. The video is accompanied 
by audio, as is typical in motion pictures. The computer 
is also connected in standard fashion to a network con- 
necting device 1004 such as ethernet, token ring or tel- 
ephone modem, which allows access to the World Wide 
Web. 

The video is contained inside a video display win- 
dow 1010. The window is bordered in standard fashion, 
to allow for moving it or resizing it by utilizing the mouse 
in standard fashion. The top border is a standard panel 
bar 1020 with an active file button 1021 which, when 
activated by pointing the cursor at it and clicking with 
the mouse, displays a menu for initiating actions such 
as exiting the video program; an active button 1022 
which, when activated, displays an options menu; and 
active buttons 1023 for miniaturizing the display window 
to an icon on the screen or on a control bar (as in Win- 
dows 95), for fast resizing between normal size to full 
screen size, and for fast termination of the video pro- 
gram. Such button configurations are standard. Under- 
neath the window is a panel bar 1030 which contains 
active regions (buttons) 1040 for controlling typical vid- 
eo functions as play and stop/pause, and an active slider 



1050 for controlling random access to temporal loca- 
tions in the video. Such configurations are standard in 
the art. 

The displayed video has been encoded with em- 
s bedded hyperlinks. These are certain regions 1 080 with- 
in the video display window for which at certain time in- 
tervals special information has been encoded in the vid- 
eo data stream that point to html files in various URLs 
which are connected to the computer network. Such re- 
10 gion 1080 during such time period is called a hot-link. 
On the panel bar 1 030 is an active light region 1 060. 
During those time periods in which there are no hot-links 
encoded in the video, the light region is green. During 
those time periods in which there are hot-links in the vid- 
is eo, the light region is red. If the cursor is inside the dis- 
play window while there is a hot-link in the video, that 
region is highlighted by either a color change or an easily 
observed border; the option is offered in the options 
menu. 

20 |f a user clicks the left button of the mouse while the 
cursor 1070 is inside the hot-link, a signal is transmitted 
from the computer to the encoded URL requesting the 
delivery of the linked html file back to the computer. The 
contents of the html file contain instructions which the 
25 computer executes. Typically these will be to display text 
or multimedia data on the monitor; the multimedia data 
may also include audio played out of the speakers. 
While this new data is being displayed, the video is 
paused. This is achieved either by a pause command 
30 on the video player if the video data has already been 
downloaded to a local file, or (more typically) by trans- 
mitting a pause request to the URL from which the hot 
video is being streamed, and having that location issue 
a pause command to stop the streaming, if the originat- 
es ing URL does not support a pause command, then the 
video continues. The user has an option to miniaturize 
the video display window while he is viewing the newly 
displayed information. At some later time, the user may 
resume the hot video by pointing to the play button in 
40 1 040 with the cursor and clicking a button on the mouse. 
If the viewer chooses to view the video in full screen 
mode (which he can do by clicking the resize button in 
1023), then the active light region is embedded in the 
video display window (now the entire display window) 
45 at the bottom right comer. 

At any time period, more than one hot-link may be 
present in the video, and they are simultaneously visible 
if the cursor is inside the video display window. 

A second embodiment of the invention is schemat- 
ic ically shown in Fig. 2. The configuration is similar to that 
of Fig. 1 , with objects (2xxx) of Fig. 2 corresponding to 
objects (Ixxx) of Fig. 1 . An object corresponding to 1060 
of Fig. 1 is not present in Fig. 2 . Objects 2009 and 2100 
of Fig 2 have no corresponding objects in Fig 1 . Object 
55 2009 is a horizontal indicator bar and object 2100 is a 
vertical indicator bar. When a hot -link is present in the 
displayed video, the X-coordinate of the center of the 
hot-link is indicated on the horizontal indicator bar di- 
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rectly below it by a bright dot, and the Y-coordinate of 
the center of the hot-link is indicated on the vertical in- 
dicator bar directly to its right by a bright dot. As in the 
previous embodiment, if the cursor is inside the display 
window while there is a hot-link in the video, that region 
is highlighted by either a color change of an easily ob- 
served border; the option is offered in the options menu. 
The user accesses the html file corresponding to the hot 
region in the same manner as in the first embodiment. 

Since the indicators of the second embodiment pin- 
point the center of the hot-link, there is an option also 
accessible via the option menu for eliminating the hot- 
link highlighters inside the video display window. The us- 
er can still quickly move the cursor to the appropriate X- 
Y coordinate position within the video display window 
and click on the hot-link. Since hot regions typically cor- 
respond to recognizable image segments, such as peo- 
ple, faces, animals, objects, the user need not position 
the cursor exactly at the X-Y coordinates to be within 
the hot-link region. Another option within the options 
menu is that the cursor itself is changed when it is inside 
the hot-link. This may be a color change or an addition 
of a question mark next to the typical cursor, or other 
change. 

Another option accessible via the options menu is 
that the horizontal and vertical indicator bars exhibit via 
lighted horizontal and vertical line segments the corre- 
sponding support intervals of hot-link. Thus, if the hot- 
link is a rectangle, then the horizontal bar is lighted along 
the entire interval directly below the hot-link, and the ver- 
tical bar is lighted along the entire interval directly to the 
right of the hot region. If the hot-link is an ellipse, the 
lighted intervals on the indicator bars may correspond 
to either the largest rectangle whose sides are parallel 
to the indicator bars and which lies either completely in- 
side the ellipse of the smallest rectangle whose sides 
are parallel to the indicator bars and which contains 
completely the ellipse. Multiple hot-links in a single 
frame may have regions whose x-coordinates or y-co- 
ordinates overlap. Various options for user interfaces to 
convey this information on the indicator bars are envi- 
sioned, and follow directly from the invention herein. 

It will be observed that the invention provides the 
user the ability to view a hot video without intrusion of 
hot-link indications inside the video window as well as 
the ability to view the video with indicators inside the 
video window visible. The choice can be dynamically in- 
voked by simply moving the cursor out of or into the vid- 
eo display window. When the user is viewing in a non- 
intrusive mode, the light indicator provides information 
as to the presence of a hot region, alerting the viewer 
that if he so chooses, he can quickly determine the lo- 
cation of the hot-link by moving the cursor inside the dis- 
play window. 

Creation of hot video content is achieved in the fol- 
lowing way. An MPEG encoded video source is used for 
this particular embodiment (D. LeGail, MOEG: A video 
Compression Standard for Multimedia Applications, 



Communications of the ACM, Vol. 34. No. 4, April 1 994). 
The MPEG system layer contains packets of video, au- 
dio, and user specified data (International Organization 
for Standardization, ISO/IEC JICI/SC291/WG11, Cod- 
5 ing of Motion Pictures and Associated Audio, N0272, 
June 10, 1 994). The user specified data will contain the 
hot-link information. Hot-link information for a given 
frame in the video will be encoded in a packet which 
immediately precedes the packet which contains the be- 

10 gtnnings of the video data for that frame. A hot video 
editing system comprising an MPEG playback system 
is used. It can play back the MPEG video in real-time 
mode, slow mode, or single-frame-at-a-time mode. It al- 
so has editing tools for inserting the hot link information. 

15 At any desired frame, the user freezes the playback at 
that frame. Using the editing tools, the user indicates the 
desired region which is to be hot. After the region is in- 
dicated, the user labels the region with a pointer to a 
URL. A fist of URLs for the particular video sequence is 

20 maintained, and displayed along the frozen single 
frame. If the desired URL is already in the list, the user 
simply has to click on it with a mouse to achieve the la- 
beling. If this URL is new, the user has to enter it into 
the list and then point the hot-link to it. The editing sys- 

25 tern inserts this information into the user data packet, as 
discussed above. 

An alternate embodiment for the creation of hot 
MPEG video, which is extensible to other video formats, 
is the following. A hot video editing system is used, as 

30 above. This time, instead of embedding the hot-link in- 
formation into the MPEG system layer data stream, a 
new type of data stream is created, which mixes the 
MPEG system layer with the hot-link information. This 
will form a new MIME type, consisting of a sequence of 

35 hot video information for a particular frame, followed by 
the video data of that particular frame. Special start code 
symbols are used to indicate the start of a hot video data 
segment; and other special start code symbols are used 
to indicate the start of a MPEG video data segment. 

40 Both hot video creation systems described above 
can be automated for faster creation of hot video. For 
example, motion estimation techniques as in MPEG en- 
coding may be used to predict the location of a hot-link 
in a frame from a previous frame. The interactive editing 

45 system allows the user to view the automatically created 
hot-link regions, and the user can make modifications if 
necessary. 

A hot video decoder is utilized by a browser when- 
ever it identifies the incoming data as hot video data; 

50 this identification is via the suffix of the data name. If the 
data is encoded using the first embodiment, with hot- 
link information embedded in the MPEG data stream, 
then the hot video decoder is an MPEG decoder which 
also understands the data provided by the user-speci- 

55 fied data packets. The decoder determines start of vid- 
eo, audio or hot-link info from the packet start codes. It 
then utilizes these three sets of information, the hot vid- 
eo data to determine the active regions and the URLs 
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to which they point, and the MPEG video and audio data 
to synchronize and display the audio and video. If the 
data is encoded using the second embodiment, the de- 
coder determines start of video or hot-link info data from 
the start code headers. It then utilizes the two sets of 
information, the hot video data to determine the active 
regions and the URLs to which they point, and the 
MPEG audio/video data to playback the video. 

A hot video decoder is created as follows. Using Mi- 
crosoft's Visual Basic, one creates a main form which 
looks like the one in Figure 1 . It contains a video window, 
plus buttons, a slider, and a color indicator. The default 
color of the color indicator is green. The decoder utilizes 
an MPEG decoder which decodes data coming either 
from file or streaming data. The decoder also interprets 
user packet data, which contains the information regard- 
ing time and place of hot link regions in the video. The 
decoder sets up an array for hot link region information. 
The array contains the coordinates of the hot link re- 
gions. The decoder also sets up a hot-link flag; if there 
is no hot link region, the flag is set to 0; if there is a hot 
link region then the flag is set to 1 . The decoder polls 
the flag every 33 milliseconds. If the flag is 1 , indicating 
the presence of a hot link region, the color indicator turns 
red. The decoder then checks the hot link region array 
to determine the position of the hot link region. It also 
checks the position of the cursor. If the cursor is inside 
the video window, then a border is displayed around the 
hot link regions. If furthermore the cursor is inside the 
hot link region, then the cursor shape is changed. If a 
user clicks the mouse while the cursor is inside a hot 
link indicator, then the linked page is displayed on the 
computer monitor, and the video is paused. If the hot 
link flag is 0, and if the color indicator was red, it now 
turns green; if there existed hot link regions, then all hot 
link region borders are removed; and if there was a spe- 
cial shaped cursor inside a hot link region, then the cur- 
sor resumes its default shape. Otherwise, the color in- 
dicator remains green, and the video proceeds undis- 
turbed. The means of achieving the above described 
processes are standard in the art. 



2. A method as claimed in claim 1 , characterized in 
that the mentioned step of displaying an indication 
of the presence of the hot link region (1080, 2080) 
in the video presentation comprises the further step 

s of altering a visual attribute of at least a portion of 
the hot link region when a user manipulate cursor 
is within the first portion (1010, 2010). 

3. A method as claimed in claim 2, characterized in 
10 that the mentioned step of altering a visual attribute 

of at least a portion of the hot link region (1080, 
2080) can be extended to the method of direct dis- 
playing the hot link region in the first portion (1010, 
2010). 

15 

4. A method as claimed in claim 1 , characterized in 
that the mentioned step of displaying an indication 
of the presence of the hot link region (1080, 2080) 
in the video presentation comprises the further step 

20 of displaying an indication (1 060, 2009, 21 00) of the 
presence of the hot link region in the video presen- 
tation on a second portion (1030, 2030) of the dis- 
play screen. 

25 s. A method as claimed in claim 4, characterized in 
that the method further comprises the step of alter- 
ing a visual attribute of at least a portion of the hot 
link region (1080, 2080) when a user manipulate 
cursor is within the first portion (1010, 2010). 

30 

6. A method as claimed in claim 4, characterized in 
that the method further comprises the step of dis- 
playing on the second portion (1030, 2030) an indi- 
cator (2009, 2100) of the location of the hot link re- 

35 gion (1080, 2080). 

7. A method as claimed in claim 4, characterized in 
that the method further comprises the step of dis- 
playing on the second portion (1030, 2030) an indi- 

40 cator (2009, 2100) of the location and size of the 
hot link region (1080, 2080). 



Claims 

45 

1 . A method for indicating the location of time depend- 
ent video hot links to a user, comprising the steps of: 

displaying a motion video presentation on a first 
portion (1010, 2010) of a display device (1000, so 
2000), the video presentation including a hot 
link region (1080, 2080) which can be selected 
by a user to link to different content; 

displaying an indication (1060, 2009, 2100) of 55 
the presence of the hot link region in the video 
presentation. 
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